In order to solve the problem that the traditional cascaded Convolutional Neural Network (CNN) has low accuracy of feature point localization in left ventricular ultrasound image, an improved cascaded CNN with region extracted by Faster Region-based CNN (Faster-RCNN) model was proposed to locate the left ventricular endocardial and epicardial feature points in ultrasound images. Firstly, the traditional cascaded CNN was improved by a structure of two-stage cascaded. In the first stage, an improved convolutional network was used to roughly locate the endocardial and epicardial joint feature points. In the second stage, four improved convolutional networks were used to fine-tune the endocardial feature points and the epicardial feature points separately. After that, the positions of joint contour feature points were output. Secondly, the improved cascaded CNN was merged with target region extraction, which means that the target region containing the left ventricle was extracted by the Faster-RCNN model and then was sent into the improved cascaded CNN. Finally, the left ventricular contour feature points were located from coarse to fine. Experimental results show that compared with the traditional cascaded CNN, the proposed method is much more accurate in left ventricle feature point localization, and its prediction points are closer to the actual values. Under the root mean square error evaluation standard, the accuracy of feature point localization is improved by 32.6 percentage points.